Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Supavisor V2 #408

Open
wants to merge 98 commits into
base: main
Choose a base branch
from
Open

Supavisor V2 #408

wants to merge 98 commits into from

Conversation

abc3
Copy link
Member

@abc3 abc3 commented Jul 31, 2024

In this PR, Supavisor has been adapted for Erlang 27 and Elixir 1.17. The metrics backend has been changed from PromEx to Peep, and the most significant change is the introduction of ProxyHandler for processing transaction mode.

The ProxyHandler first authenticates the incoming connection through auth_query. If there is no connection pool, it creates one and also launches a Ranch on a free port, saving the port and host in the corresponding Syn record.

If the pool is on the same node as the client connection, we locally change the owner of the TCP socket for each transaction to the direct connection with the database. After the transaction is completed, the client process returns ownership of the TCP socket to the DbHandler.

If the pool is on a different node, the client process connects to the Ranch of the corresponding pool on the other node, switches to session mode, and forwards everything (works like proxy). The local Ranch operates the same way as when the client connection is on the same node as the connection pool.

With this handler, erldist is used only for synchronizing Syn. Forwarding query data between nodes is done via TCP sockets (Ranch)

hauleth and others added 29 commits July 2, 2024 09:30
Reason for that is that `a || b` will expand to:

```
case a do
  val when val in [false, nil] -> b
  val -> val
end
```

Which is harder to optimise by the compiler. `or/2` works only with
booleans, so in places where we do not care about returned value or
`nil` cases, it is better to use `or/2`.
* ft: improve socket metrics gathering

As we are only interested in just 2 of the values from the socket stats, fetch
only these. This should reduce time required for processing data sent between
port and application. It also simplifies pattern matching and data extraction.

* ft: gather metrics from other nodes with low priority
* chore: update Benchee

* chore: unlock telemetry_metrics

* chore: remove unused libraries from lockfile

* chore: update Rustler and Rust dependencies

* chore: update direct dependencies

* chore: upddate Phoenix.LiveView
Supavisor is not using any HTML views, it do not offer any HTML-based
UI, etc. so these functionalities aren't needed for anything.
I have used [`typos`][typos] tool to find all typos in source code over
the codebase.

[typos]: https://github.com/crate-ci/typos
* chore: add Nix Flake to project

Currently it defines only development environment for Supavisor. In the
future the plan is to add package definition as well as NixOS module to
the Flake. This will simplify deployment on NixOS platforms as well as
should help with spawning virtual machines with Supavisor running on
them.

* ft: add derivation for Supavisor
All changes required by our code are merged to mainline, so we do not
need to use custom fork anymore.
@abc3 abc3 changed the title Supavisor V2 [WIP] Supavisor V2 Jul 31, 2024
abc3 and others added 23 commits October 11, 2024 20:41
feat: shorter retry connect timeout for proxy mode
This will raise if there are some pending changes to the `mix.lock`.
This is to ensure, that `mix.lock` was uploaded after dependency
changes.
* fix: update max clients error in session mode
That tests are constantly causing us problems in CI, but as it is test
for internal functionality that do not affect users directly. So to
streamline testing disable running these tests in default run. To run
them one can use

    mix test --include flaky
These plugs weren't used at all, as we do not serve any static assets nor we
need live reload for anything (as we do not have any form of web UI).
Handle errors during authentication query gracefully and shutdown
the client process to not leave the hanging connections.
* test: increase pool size in external test suite to match nano instances

* test: make timeouts configurable

* test: disable prepared transactions test, as these are disabled for security reasons

* test: disable cursor tests in transaction mode

* test: disable custom socket test

* test: disable MAX_PARAMETERS_EXCEEDED test

* test: disable query format errors test

* test: run connection reserving test only in session mode

* test: make Postgres.js more verbose

* test: ensure connections with different parameters are available where needed

* test: fix typo in test name
This allows us to use this piece of code in function head in pattern
match. This reduces places where the magic value is required and make
the code a little bit neater and cleaner.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants